This invention relates to a video security system having components physically located at a premise being protected and components located at a central station from which a number of premises can be simultaneously monitored; and, more particularly, to video security system which perform a continuous surveillance of each of the premises being monitored, which reliably detects and reports intrusions of any premises to the central station for a system operator to take appropriate action including reporting the intrusions to local authorities at the site where the premises is located, and which eliminates reporting of false and unwanted alarms whereby the operator is assured that any reported alarm is an actual intrusion. Further, the video security system has the unique capability of utilizing its video camera and associated capabilities as a new type of motion sensor in conjunction with conventional alarm system functions.
Conventional security systems typically protect a building using make/break contacts strategically placed at doors, windows, and other potential entry points. When a contact is broken and an alarm is sounded or relayed to a central control station located within the building, nearby to the building, or remotely to a central control station of the security company. Besides make/break sensors, security companies also use P.I.R. (passive infra red) sensors which sense heat differences caused by animate objects such as humans or animals. Also used are vibration sensors which, when placed upon a window for example, detect when the window is broken, and radio frequency (rf), radar, and microwave sensors, as well as laser sensing. As with the make/break sensors, when any one of the sensors indicates a detection, a system alarm is indicated. A trouble indication is also given if an alarm unit for the building to which the sensors are connected senses that a path to a sensor is interrupted or broken.
With current alarm systems, all that the receiver of an alarm, whether at a local or remote central station, knows is that an alarm has occurred. The mere fact of an alarm provides no indication as to its cause. The system operator has no other knowledge by which he can determine if an alarm signals the presence of a real intruder, or if it is a false alarm. Sensors are notorious for going off during inclement weather (they are sensitive to large electromagnetic fields such as occur during lightning conditions, etc.). Such an erroneous condition is properly referred to as a false alarm. When certain sensors are triggered as by the presence of animals within their precincts, an unwanted alarm is generated. Regardless of why they occur, all false and unwanted alarms detrimentally effect the efficiency and operation of a security system. For convenience, all false and unwanted alarms will be generally referred to as false alarms.
Many criteria determine whether or not an alarm condition exists. For example, when a person opens a door monitored by a sensor, a potential alarm condition is created. However, an alarm system typically has a keypad or other coded system control by which, if an appropriate entry is made within a prescribed period of time, signifies that the alarm condition is not to be acted upon. Rather, the entrant is someone authorized to enter the premises. Further, the class of intruder (human, animal, etc.) may be perfectly acceptable in one set of circumstances, but not so in another. The common situation is one where an intruder is a human, and his presence results in an alarm being given. However, if the intruder is a cat or a dog, for example, giving an alarm is inappropriate and would be considered a false alarm. In an aviary, on the other hand, the presence of a human might be perfectly acceptable, but the presence of a cat or dog should result in an alarm being given. Or, in an area where food is stored, the presence of a human may again be perfectly acceptable, but the appearance of mice or rats should trigger an alarm. There is currently no alarm system capable of classifying intruders, discriminating between acceptable and non-acceptable classes of intruders, and providing an appropriate alarm when the presence of an intruder from a non-acceptable class is detected.
False alarms plague the security system industry. While the situation is annoying when a false alarm is relayed to a local monitoring station, it is exacerbated when the alarm is relayed 2,000 miles or more to a security company's central station or monitoring facility which monitors a large number of widely separated facilities. Here, operators must use their experience of the particular circumstances surrounding the alarm (i.e., local weather conditions, past occurrences at that particular site, etc.), to determine whether or not the alarm is real. If their knowledge and experience tells them the alarm signifies an actual intrusion, they then relay the alarm to the local police for that site so the police can conduct a further investigation.
There are numerous examples of an alarm system being rendered ineffective simply because an operator at a control station had no insight into the facility where the alarm system was installed and from which an alarm emanated. As one example, an alarm was triggered by a cat left in a residence. The police were called but discovered nothing, not even the cat, because the cat hid from strangers. After repeated visits over a period of several hours, the police finally refused to investigate further. From that point on, the residence was essentially not protected. Over time, situations resulting from false alarms have continued to worsen. Now, police often require confirmation or evidence of an intrusion before investigating, or else they give priority to those situations where they have greater certainty an intrusion has occurred. Furthermore, in many locales, if the authorities investigate the report of an alarm and discover nothing, they will send the security company requesting the investigation a bill for their services. Security system companies have addressed this issue by providing an audio (or "listening-in") capability to the system. This enables the monitor to hear actual movement on the premises (the sound of voices, glass breaking, cabinets or drawers being opened, etc.,) with this information also being relayed to the authorities.
In response to this situation, the security industry has begun using video cameras to monitor premises. Use of cameras solves the problem of not just reacting to a make/break contact. The shortcomings with camera surveillance are that one needs a constant communications channel between the sensor (camera) and the operator at the monitoring site. Some alarm systems have combined video with another sensing mechanism, I.R., for example, so actuation of the video is controlled by the other sensor first sensing the presence of an intruder.
If video is continuously required for a properly functioning system, a communications channel having sufficient bandwidth to transmit video must be constantly connected between the site and the monitoring station, from the time the alarm system is energized until it is turned-off. Since monitoring periods often exceed 12 hours, communication costs are high. To help control communication costs, the cameras used are often slow scan cameras whose output is compressed onto POTS (plain old telephone system) lines (typically using 28.8 k modems) with transmission rates of 1 frame of video over a 1-5 second interval.
Importantly, when monitoring continuous video, most of the time, the operator will see nothing out of the ordinary. Yet, the operator must maintain a constant vigilance. This is a serious problem because it has been estimated that after watching a security system camera observing an unchanging scene for as little as 5 minutes, an operator's performance diminishes rapidly to the point where the operator is essentially ineffective after 30 minutes. As a consequence, the only real advantage of video monitoring is that should an intrusion occur, and should the operator notice it, then the relayed information sent to the local police gets high priority because of the certainty of the situation. Apart from this advantage, the deficiencies of such a system are that it is very labor intensive, operator efficiency is usually very low, and communications costs are very high.
As an adjunct to video surveillance, security companies also employ guards who are either stationed on the premise, or tour a number of premises using a patrol car to move from one site to another. Guards are expensive. Sophisticated criminals know they can determine the guards' routine and can plan break-ins at those times when a guard is somewhere else on the premise, or at a another site altogether.
The video security system of the present invention overcomes these problems by, inter alia, providing an alarm system operator indications of an intrusion (whether by humans or an unknown source) together with visual images of the intrusion. This keeps the operator from "crying wolf" because of false alarms, and provides a level of surveillance not obtainable even using guards. It is now possible, as described hereinafter, to relay definitive information to the local police of an intrusion, as well as capture, maintain, and transmit images of the intrusion to the police or other authorities. It is also possible to minimize, if not eliminate false alarms. Finally, it is possible to remotely perform a video guard tour of a premises, at any time, during unusual or extreme weather conditions, and without disturbing the normal routine of a premises.
In co-pending U.S. patent application Ser. Nos. 08/772,731; 08/757,838; 08/771,991; and 08/772,595; the teachings of which are incorporated herein by reference, there is described a system and method to i) continuously viewing a scene to detect motion, ii) to identify and classify the cause of the motion, iii) to selectively report the presence of an intruder with a very low probability of false alarms and a high probability of detection; and iv) to provide a method for viewing a scene, producing an image of the scene, authenticating that image, and relaying the authenticated image from the protected site to a remote, viewing site. As described in these applications a reference scene (reference image) is established and an image from the present scene (current image) is compared to that reference. It is then determined whether any differences exist between the present and reference images. If the contents of the two images markedly differ, the result is interpreted as an intrusion of some kind having occurred. The detection process includes comparing, on a pixel by pixel basis, the current image with the reference image to obtain a difference image. In accordance with the process, any non-zero pixel in the difference image indicates the possible presence of an intrusion (after image artifacts such as noise, aliasing of the video, and movement within the scene not attributable to a life form (animal or human) such as the hands of a clock, screen savers on computers, oscillating fans, etc., have been accounted for). Because the system and method use an absolute difference technique with pixel by pixel subtraction, the process, as described in co-pending application Ser. No. 08/772,595, is sensitive to surface differences between the images, is insensitive to light-on-dark or dark-on-light changes, and thus is very sensitive to any intrusion within the scene. Furthermore, each pixel represents a gray level measure of the scene intensity that is reflected from that part of the scene. Gray level intensity can change for a variety of reasons. The most important of these is a new physical presence at that particular part of the scene. The ability to make this determination, in accordance with the teachings set forth in these co-pending applications, removes from the human operator of the alarm system the initial responsibility of determining whether an intrusion results from a new human presence or an unknown source. This eliminates the need for the human operator to continuously monitor images from the sites being protected; and, in turn, helps the operator maintain a high level of vigilance. The detection and recognition for each premises is conducted on-site at that premise and the system is always vigilant. There is also now no need for a continuous, high level communication path between the premises and a central station of the system. When an intrusion is detected and a path is established, high frame rate and high quality video is transferred from the site to the operator so the operator can evaluate the video and concur with the evaluation an intrusion has occurred. Further, once an intrusion is detected, high resolution samples ("snapshots") of the video are taken and supplied to the security system operator at the central station. These samples are transferred using lossless compression techniques and are authenticated for later admittance into court for prosecution purposes.
Efforts have previously been made to incorporate object recognition, including recognition of humans, whose presence is detected or sensed in an image, into some type of control unit. U.S. Pat. No. 5,305,390 to Frey et al., teaches automatic recognition and classification of persons or objects as they pass through a doorway or entrance. The intrinsic sensor is an active laser beam, and the system of Frey et al. operates by measuring the height of an object passing through an aperture (doorway) to classify the object as a person or not. Therefore, the system is a height discriminator rather than an object recognition or classification system. Thus, for example, if a person crawls through the aperture, they will probably be designated as a non-human. Also, the intruder must be at a known location (door, window, etc.), or otherwise the detection scheme does not work.
U.S. Pat. No. 5,289,275 to Ishii et al., is directed to a surveillance monitoring system using image processing for monitoring fires and thefts. The patent teaches use of a color camera for monitoring fires and a method of comparing the color ratio at each pixel in an image to estimate the radiant energy represented by that pixel. A resulting ratio is compared to a threshold with the presence of a fire being indicated if the threshold is surpassed. A similar technique for detecting the presence of humans is also described. The patent teaches the use of image processing together with a camera to detect the presence of fires and abnormal objects.
U.S. Pat. No. 4,697,077 to Yausa et al. also teaches use of a camera to detect the presence of an object. Once an anomaly is detected because of differences in the comparison of an original and a later image, the system automatically dials and sends a difference image, provided the differences are large enough, to a remote site over a telephone line. At the remote site, the image is viewed by a human. While teaching some aspects of detection, Yausa et al. does not go beyond the detection process to attempt and use image processing to recognize that the anomaly is caused by a human presence.
U.S. Pat. No. 4,257,063 which is directed to a video monitoring system and method, teaches that a video line from a camera can be compared to the same video line viewed at an earlier time to detect the presence of a human. However, here, the detection device is not a whole image device, nor does it make any compensation for light changes, nor does it teach attempting to automatically recognize the contents of an image as being derived from a human. Similarly, U.S. Pat. No. 4,161,750 teaches that changes in the average value of a video line can be used to detect the presence of an anomalous object. Whereas the implementation is different from the '063 patent, the teaching is basically the same.
All of these previous attempts at recognition have certain drawbacks, whether the type of imaging, method of processing, etc., which would result in either an alarm not being provided when one should, or in false alarms being given. The video security system of the present invention overcomes these problems or shortcomings to reliably provide accurate indications of an intrusion, by one of an appropriately designated class, in an area being monitored by the system.